from sklearn_benchmarks.reporting.hp_match import HPMatchReporting

scikit-learn vs. ONNX Runtime

reporting = HPMatchReporting("onnx", config="config.yml")
reporting.run()

KNeighborsClassifier_brute_force

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: algorithm=brute.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	KNeighborsClassifier_brute_force	predict	100000	1000	100	16.921	0.009	0.0	0.017	-1	1	0.663	2.034	0.029	8.319	0.119	See	See
1	KNeighborsClassifier_brute_force	predict	100000	1000	100	16.921	0.009	0.0	0.017	-1	1	0.663	0.025	0.003	684.480	82.876	See	See
2	KNeighborsClassifier_brute_force	predict	100000	1	100	0.313	0.010	0.0	0.313	-1	1	1.000	2.034	0.029	0.154	0.005	See	See
3	KNeighborsClassifier_brute_force	predict	100000	1	100	0.313	0.010	0.0	0.313	-1	1	1.000	0.025	0.003	12.654	1.585	See	See
4	KNeighborsClassifier_brute_force	predict	100000	1000	100	16.837	0.026	0.0	0.017	-1	5	0.757	3.096	0.075	5.437	0.131	See	See
5	KNeighborsClassifier_brute_force	predict	100000	1000	100	16.837	0.026	0.0	0.017	-1	5	0.757	0.026	0.002	656.125	58.761	See	See
6	KNeighborsClassifier_brute_force	predict	100000	1	100	0.316	0.010	0.0	0.316	-1	5	1.000	3.096	0.075	0.102	0.004	See	See
7	KNeighborsClassifier_brute_force	predict	100000	1	100	0.316	0.010	0.0	0.316	-1	5	1.000	0.026	0.002	12.304	1.169	See	See
8	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.420	0.030	0.0	0.017	1	100	0.882	2.072	0.008	8.407	0.034	See	See
9	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.420	0.030	0.0	0.017	1	100	0.882	0.023	0.003	745.202	92.908	See	See
10	KNeighborsClassifier_brute_force	predict	100000	1	100	0.307	0.006	0.0	0.307	1	100	1.000	2.072	0.008	0.148	0.003	See	See
11	KNeighborsClassifier_brute_force	predict	100000	1	100	0.307	0.006	0.0	0.307	1	100	1.000	0.023	0.003	13.135	1.655	See	See
12	KNeighborsClassifier_brute_force	predict	100000	1000	100	16.848	0.064	0.0	0.017	-1	100	0.882	3.109	0.085	5.420	0.150	See	See
13	KNeighborsClassifier_brute_force	predict	100000	1000	100	16.848	0.064	0.0	0.017	-1	100	0.882	0.027	0.002	625.782	51.593	See	See
14	KNeighborsClassifier_brute_force	predict	100000	1	100	0.309	0.006	0.0	0.309	-1	100	1.000	3.109	0.085	0.099	0.003	See	See
15	KNeighborsClassifier_brute_force	predict	100000	1	100	0.309	0.006	0.0	0.309	-1	100	1.000	0.027	0.002	11.484	0.974	See	See
16	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.102	0.157	0.0	0.017	1	5	0.757	2.069	0.009	8.267	0.084	See	See
17	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.102	0.157	0.0	0.017	1	5	0.757	0.022	0.000	766.058	18.035	See	See
18	KNeighborsClassifier_brute_force	predict	100000	1	100	0.308	0.005	0.0	0.308	1	5	1.000	2.069	0.009	0.149	0.003	See	See
19	KNeighborsClassifier_brute_force	predict	100000	1	100	0.308	0.005	0.0	0.308	1	5	1.000	0.022	0.000	13.794	0.387	See	See
20	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.388	0.176	0.0	0.017	1	1	0.663	1.193	0.005	14.572	0.159	See	See
21	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.388	0.176	0.0	0.017	1	1	0.663	0.021	0.000	841.854	16.128	See	See
22	KNeighborsClassifier_brute_force	predict	100000	1	100	0.310	0.004	0.0	0.310	1	1	1.000	1.193	0.005	0.260	0.003	See	See
23	KNeighborsClassifier_brute_force	predict	100000	1	100	0.310	0.004	0.0	0.310	1	1	1.000	0.021	0.000	15.014	0.310	See	See
24	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.737	0.018	0.0	0.004	-1	1	0.896	1.920	0.036	1.947	0.038	See	See
25	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.737	0.018	0.0	0.004	-1	1	0.896	0.009	0.002	419.097	109.688	See	See
26	KNeighborsClassifier_brute_force	predict	100000	1	2	0.241	0.004	0.0	0.241	-1	1	1.000	1.920	0.036	0.126	0.003	See	See
27	KNeighborsClassifier_brute_force	predict	100000	1	2	0.241	0.004	0.0	0.241	-1	1	1.000	0.009	0.002	27.068	7.095	See	See
28	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.695	0.008	0.0	0.004	-1	5	0.922	3.075	0.051	1.202	0.020	See	See
29	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.695	0.008	0.0	0.004	-1	5	0.922	0.010	0.002	382.730	77.015	See	See
30	KNeighborsClassifier_brute_force	predict	100000	1	2	0.240	0.004	0.0	0.240	-1	5	1.000	3.075	0.051	0.078	0.002	See	See
31	KNeighborsClassifier_brute_force	predict	100000	1	2	0.240	0.004	0.0	0.240	-1	5	1.000	0.010	0.002	24.886	5.027	See	See
32	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.748	0.007	0.0	0.004	1	100	0.929	1.992	0.006	1.882	0.007	See	See
33	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.748	0.007	0.0	0.004	1	100	0.929	0.003	0.000	1263.004	103.334	See	See
34	KNeighborsClassifier_brute_force	predict	100000	1	2	0.241	0.004	0.0	0.241	1	100	1.000	1.992	0.006	0.121	0.002	See	See
35	KNeighborsClassifier_brute_force	predict	100000	1	2	0.241	0.004	0.0	0.241	1	100	1.000	0.003	0.000	81.043	6.790	See	See
36	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.750	0.010	0.0	0.004	-1	100	0.929	3.012	0.047	1.245	0.020	See	See
37	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.750	0.010	0.0	0.004	-1	100	0.929	0.010	0.003	357.668	91.766	See	See
38	KNeighborsClassifier_brute_force	predict	100000	1	2	0.240	0.005	0.0	0.240	-1	100	1.000	3.012	0.047	0.080	0.002	See	See
39	KNeighborsClassifier_brute_force	predict	100000	1	2	0.240	0.005	0.0	0.240	-1	100	1.000	0.010	0.003	22.851	5.881	See	See
40	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.622	0.008	0.0	0.004	1	5	0.922	1.983	0.005	1.827	0.006	See	See
41	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.622	0.008	0.0	0.004	1	5	0.922	0.003	0.000	1231.886	139.557	See	See
42	KNeighborsClassifier_brute_force	predict	100000	1	2	0.239	0.003	0.0	0.239	1	5	1.000	1.983	0.005	0.120	0.002	See	See
43	KNeighborsClassifier_brute_force	predict	100000	1	2	0.239	0.003	0.0	0.239	1	5	1.000	0.003	0.000	81.172	9.255	See	See
44	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.767	0.050	0.0	0.004	1	1	0.896	1.097	0.006	3.435	0.049	See	See
45	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.767	0.050	0.0	0.004	1	1	0.896	0.002	0.000	2112.310	271.172	See	See
46	KNeighborsClassifier_brute_force	predict	100000	1	2	0.237	0.005	0.0	0.237	1	1	1.000	1.097	0.006	0.217	0.004	See	See
47	KNeighborsClassifier_brute_force	predict	100000	1	2	0.237	0.005	0.0	0.237	1	1	1.000	0.002	0.000	133.171	17.195	See	See

KNeighborsClassifier_kd_tree

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: algorithm=kd_tree.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	120.914	0.000	0.0	0.121	-1	1	0.929	0.536	0.005	225.740	2.133	See	See
1	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	120.914	0.000	0.0	0.121	-1	1	0.929	0.003	0.001	41242.490	8832.226	See	See
2	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.627	0.185	0.0	2.627	-1	1	1.000	0.536	0.005	4.905	0.349	See	See
3	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.627	0.185	0.0	2.627	-1	1	1.000	0.003	0.001	896.055	202.048	See	See
4	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	122.302	0.000	0.0	0.122	-1	5	0.946	0.994	0.010	123.053	1.259	See	See
5	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	122.302	0.000	0.0	0.122	-1	5	0.946	0.004	0.001	27751.805	4433.762	See	See
6	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.575	0.079	0.0	2.575	-1	5	1.000	0.994	0.010	2.590	0.084	See	See
7	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.575	0.079	0.0	2.575	-1	5	1.000	0.004	0.001	584.189	95.055	See	See
8	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	132.262	0.000	0.0	0.132	1	100	0.951	5.847	0.017	22.622	0.067	See	See
9	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	132.262	0.000	0.0	0.132	1	100	0.951	0.005	0.000	25410.793	1132.827	See	See
10	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.583	0.064	0.0	2.583	1	100	1.000	5.847	0.017	0.442	0.011	See	See
11	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.583	0.064	0.0	2.583	1	100	1.000	0.005	0.000	496.165	25.281	See	See
12	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	127.521	0.000	0.0	0.128	-1	100	0.951	3.415	0.032	37.341	0.354	See	See
13	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	127.521	0.000	0.0	0.128	-1	100	0.951	0.006	0.001	19747.616	2521.160	See	See
14	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.596	0.090	0.0	2.596	-1	100	1.000	3.415	0.032	0.760	0.027	See	See
15	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.596	0.090	0.0	2.596	-1	100	1.000	0.006	0.001	402.051	53.181	See	See
16	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	128.932	0.000	0.0	0.129	1	5	0.946	1.764	0.013	73.095	0.524	See	See
17	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	128.932	0.000	0.0	0.129	1	5	0.946	0.002	0.000	71966.817	18231.637	See	See
18	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.581	0.083	0.0	2.581	1	5	1.000	1.764	0.013	1.463	0.048	See	See
19	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.581	0.083	0.0	2.581	1	5	1.000	0.002	0.000	1440.799	367.963	See	See
20	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	132.215	0.000	0.0	0.132	1	1	0.929	0.923	0.010	143.189	1.548	See	See
21	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	132.215	0.000	0.0	0.132	1	1	0.929	0.001	0.000	126412.575	36831.359	See	See
22	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.591	0.072	0.0	2.591	1	1	1.000	0.923	0.010	2.806	0.084	See	See
23	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.591	0.072	0.0	2.591	1	1	1.000	0.001	0.000	2476.822	724.960	See	See
24	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.039	0.000	0.0	0.000	-1	1	0.891	0.023	0.001	1.683	0.054	See	See
25	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.039	0.000	0.0	0.000	-1	1	0.891	0.002	0.000	15.947	1.844	See	See
26	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	1	1.000	0.023	0.001	0.227	0.009	See	See
27	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	1	1.000	0.002	0.000	2.150	0.253	See	See
28	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.039	0.000	0.0	0.000	-1	5	0.911	0.024	0.001	1.603	0.042	See	See
29	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.039	0.000	0.0	0.000	-1	5	0.911	0.002	0.000	16.694	3.471	See	See
30	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	5	1.000	0.024	0.001	0.217	0.008	See	See
31	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	5	1.000	0.002	0.000	2.261	0.475	See	See
32	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.061	0.000	0.0	0.000	1	100	0.894	0.034	0.000	1.788	0.008	See	See
33	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.061	0.000	0.0	0.000	1	100	0.894	0.001	0.000	98.915	26.755	See	See
34	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	100	1.000	0.034	0.000	0.154	0.006	See	See
35	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	100	1.000	0.001	0.000	8.504	2.322	See	See
36	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.061	0.000	0.0	0.000	-1	100	0.894	0.037	0.002	1.645	0.110	See	See
37	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.061	0.000	0.0	0.000	-1	100	0.894	0.002	0.000	25.944	5.112	See	See
38	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	100	1.000	0.037	0.002	0.142	0.010	See	See
39	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	100	1.000	0.002	0.000	2.233	0.445	See	See
40	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.040	0.000	0.0	0.000	1	5	0.911	0.021	0.000	1.885	0.022	See	See
41	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.040	0.000	0.0	0.000	1	5	0.911	0.001	0.000	64.749	21.135	See	See
42	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	5	1.000	0.021	0.000	0.255	0.009	See	See
43	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	5	1.000	0.001	0.000	8.744	2.868	See	See
44	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.039	0.000	0.0	0.000	1	1	0.891	0.020	0.000	1.984	0.017	See	See
45	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.039	0.000	0.0	0.000	1	1	0.891	0.001	0.000	64.783	18.967	See	See
46	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	1	1.000	0.020	0.000	0.268	0.008	See	See
47	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	1	1.000	0.001	0.000	8.739	2.571	See	See

HistGradientBoostingClassifier_best

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: learning_rate=0.01, n_iter_no_change=10.0, max_leaf_nodes=100.0, max_bins=255.0, min_samples_leaf=100.0, max_iter=300.0.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	n_iter	iteration_throughput	latency	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	HistGradientBoostingClassifier_best	predict	100000	1000	100	0.476	0.017	300	0.002	0.000	0.824	0.114	0.003	4.177	0.187	See	See
1	HistGradientBoostingClassifier_best	predict	100000	1000	100	0.476	0.017	300	0.002	0.000	0.824	0.019	0.003	25.612	3.735	See	See
2	HistGradientBoostingClassifier_best	predict	100000	1	100	0.393	0.011	300	0.000	0.393	1.000	0.114	0.003	3.450	0.135	See	See
3	HistGradientBoostingClassifier_best	predict	100000	1	100	0.393	0.011	300	0.000	0.393	1.000	0.019	0.003	21.155	3.051	See	See